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ABSTRACT 



Instructional Research Online (InTRO) was established in 
1994 as a repository of research papers, media, and other artifacts related 
to research in instructional technology. The World Wide Web site 
(http://intro.base.org) includes sections on reviews of books and media, 
original research, links to resources in instructional technology, and 
interviews with research leaders. This paper identifies and reports on an 
analysis of the published interviews. This analysis represents what was said 
during the course of the interviews; it is a synthesis of comments, 
suggestions, and other important points made by the interviewees. Highlights 
include: (1) how to identify a research problem or question, including 

gaining confidence, reviewing the literature, and finding questions in real 
world environments; (2) the relationship between research and development, 
including blurring boundaries between work interests and research interests, 
connecting research and practice, and finding interesting projects; (3) 
choosing a mentor, including the role of the mentor and apprentice; (4) 
research methodology, including qualitative versus quantitative research, 
research traditions, and mixed methods; and (5) disseminating research, 
including the research agenda, writing research, conferences, and journals. 
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Introduction 

This is not a typical research paper. This is a paper about research. In 1994, four fairly recent doctoral 
graduates of the university of Georgia were discussing the possibilities of publishing on the World Wide Web. We 
established Instructional Research Online (InTRO) as a repository of research papers, media, and other artifacts 
related to research in Instructional Technology. Defining a direction for InTRO was, and remains an iterative 
process. The website (available: http://intro.base.org) includes sections on reviews of books and media, original 
research, links to resources in instructional technology, and interviews with research leaders. One feature that has 
helped define InTRO and its purpose has been the section on interviews with research leaders. 

These interviews are informal conversations with people who conduct research in the field of instructional 
technology. The purpose of the interviews is to gain insight into the processes and issues of research in the field of 
instructional technology by conversing with people who have significant experience engaging in research in and 
tangential to our field. 

Our practice of interviewing research leaders may have grown out of a doctoral seminar in research at the 
University of Georgia taught by Tom Reeves. The culminating assignment of the course was to focus on a research 
leader in instructional technology and to read from that person’s body of research. Analyzing this body of work, 
you could begin to see what somebody’s research agenda was, how their research had remained focused on a single 
area, shifted slightly, or shifted radically. The final phase of this assignment was to contact this person and 
interview them. All of the information was assimilated and presented back to the class. 

The practice of interviewing research leaders was a valuable one. We may have learned as much from 
talking to these people personally as we did by reading journal articles authored by them. We certainly came to 
realize that people who are regarded as leaders in research in our field can also be pleasant people who are interested 
and willing to talk about their work. Much of what was learned about research in this class was through contact 
with people on a personal level. One reason for this may be that much of what people know is not set forth in print. 
It is informal knowledge, or patterns of understanding that may not be documented, codified, or shared formally. 

Most of us will recognize that there are times when we learn as much working on projects as we learn 
sitting in classes. For example, while one may study front end analysis it is not until one does an actual analysis that 
one realizes the practical application of gathering information about an instructional environment. Work in practical 
settings helps inform, reinforce, and extend what we study. One reason for this is that when we work on projects we 
work with people who know more than we do, and we are able to tap into the informal body of knowledge that they 
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hold. -This is true in the study of research as well. Informal conversationsribout research are important in helping 
form. opinions on research, and in helping people figure out how to conduct research. 

In our interviews we were interested not only in what kind of research these people were doing, but we 
were also interested in what advice they might have to people in the field interested in conducting research. It 
remains our contention that there is much to be learned from people who conduct research. Fittingly enough, our 
First research leader interview was with Tom Reeves. 

Selection of Leaders 

In every field there are individuals who are regarded as leaders or, if you will, bellwethers for the field. 
Their work is generally recognized as important because it not only informs the field, but it also shapes the direction 
of the field. Their work may not always be agreed with, but it is recognized as important and significant. It helps to 
challenge what we know in order that we may ultimately know more. 

The selection of leaders for our interviews is an ongoing process; we continue to conduct and publish 
interviews. Our original selection of leaders to interview was certainly based on their contributions to the field, but 
other factors may have played a role in their selection as well. The InTRO editorial board started as a team of young 
researchers. While we did the initial screening without any formal selection criteria, we did work to select a balance 
of perspectives. It was and remains a goal to include people from different institutions and different research areas. 
We have included not only people conducting research in academe, but also people conducting research in industry. 

We tended to interview people who had a significant number of publications, and who had been in the field 
for more than 10 years. We looked to people who were active in professional organizations such as AECT, and we 
have striven to include a diversity of views from a diverse group of people. Given that we all have some connection 
to UGA (Susan Land, our most recent editor graduated from Florida State University but did a postdoc at UGA) our 
sample may be biased towards that institution. Additionally, there is a fair amount of self selection among our 
interviewees as a number of our requests for interviews have been declined. Consequently the people represented in 
these interviews do represent good, interesting, and challenging perspectives, though they may not represent every 
perspective at this point. It should be stated that the InTRO editorial board is always open to suggestions on 
research leaders to interview. 

Analyzing the Interviews 

The purpose of this paper is to identify and report on an analysis of the published interviews. Interviews 
are conducted via email. Typically, we review a researcher's work, and then email them a set of questions based on 
that review that we think will be of interest to our audience. Once we receive the researcher's response we then send 
a set of follow-up questions. We compile these with the original responses, ask the interviewee to review it, and 
then post it on InTRO. 

The following analysis represents a unique opportunity for the reader. All of the data we have available to 
us is available to you. This analysis represents what was said during the course of the interviews, and unlike other 
types of interviews used in research, we were not able to go back and ask for clarification after analysis had begun. 
This is not, in the strictest sense, a research article. It is a synthesis of comments, suggestions, and other important 
points made by the interviewees. We make this distinction because we feel that there are conclusions that we think 
are appropriate, but are simply not borne out in the interviews. There are conclusions that could be drawn in this 
paper based on personal experience and the interviews, but we shall refrain from making those in this paper. We 
wish to concentrate on setting forth only what can be drawn from the interviews themselves. There are obviously 
many interpretations that are possible based on the interviews. This paper and the accompanying presentation 
represents one. We invite you to draw others for yourselves. And with that invitation we also extend to you the 
invitation to formalize your interpretations and submit them to InTRO for publication. 

How to identify a problem: A question of questions 

The journey of a thousand miles is begun with a single step. This may be true, but if you are unclear on the 
direction in which your first step should be, your journey may not go that well. This is the problem many people 
have in conducting research: it is hard to identify a unique question. The interviews provide some help in this area. 
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Gaining confidence ' * ' 

Nearly everyone working on research issues has faced the problem of identifying an area in which to do 
research. There is no shortage of important questions to ask, or important issues to study according to the 
interviewees, but they do recognize that people find it hard to get started in research. Most people who find it hard 
to identify a problem may not suffer from a lack of ideas, but rather from a lack of confidence that their ideas were 
good and viable. It is suggested by our researchers that one thing that has made them “good” at research has been 
believing that their ideas were important and of interest. They have learned to analyze questions, and to “trust their 
instincts” when looking at problems in the field. One thing that has helped them “trust their instincts” has been the 
involvement of a good mentor or good colleagues who can serve as a sounding board for idea. Mentors and 
colleagues can help people gain support for their ideas and give them the encouragement they need to pursue them. 

Reviewing the literature 

There are many ways that a researcher can gain support for their ideas. One is by careful consideration of 
the existing research. One sound and practical method suggested on how to identify a research area is to think about 
the areas that you want to know something about and go read about them. As you read you may find “holes” in the 
literature, that is to say areas in the literature where significant questions are going unanswered. If you want an 
answer to a question, but there exists no answer in the literature, then that is an area of research to pursue. The 
“holes” are where you should be working. Reading and understanding the literature is critically important, as is the 
ability to read and synthesize large bodies of research. One researcher feels that the work they have done in writing 
literature reviews is the most difficult, but possibly the most important work that they have done. 

Finding studies and other articles that support your research ideas is only one reason to consider the 
literature carefully. Finding absolutely no consideration of your ideas in the literature is a fairly compelling 
indication that a pressing need for your work exists. Again, this is looking at the holes in the literature and focusing 
your work to fill those holes. As you begin to do research, your work begins to help answer important questions, but 
may not be able to answer every question. We are reminded by many research leaders that few individuals will 
make a difference with one piece of work. Collectively we contribute to the body of literature in the field. The 
knowledge base grows as more people ask and answer questions. The combined total of all of this asking and 
answering is what we know in the field of instructional technology. It is contributing to this body of knowledge that 
is important. 

Finding questions in real world environments 

Still, many people are simply trying to get an idea started. Reading the literature and finding holes in the 
problem may seem overwhelming to some. Beyond being overwhelming, it may not be practical to gain all 
questions from the literature. Looking at naturally occurring environments is one way to find out what questions are 
out there. One piece of advice is for researchers to go into educational environments, such as schools, and observe 
people using an educational product. We should look to see where they are having problems, what is making sense 
to the user, and what is confounding the user. Noting these problems can provide excellent sources for ideas in 
research. 

Combining observations and literature can help inform your reading, and help to isolate problems that 
literature can then be associated with. It is not only the observations of people using other people’s products that 
can lead to interesting research projects, but it is also observing people working with products that we develop that 
can inform research questions. 

Instructional technologists are involved in a number of efforts. In addition to reading, we may be working 
on media design and development projects. Practical experience, especially for people new to the field, provide us 
with activities that enable us to gain a better insight in to the way the field works, and what it means to work in the 
field. Many of the researchers interviewed here point to the close bond between research and development. 
Problems encountered during a development project often lead to interesting research questions. Leaders who have 
worked in industry state that their experience in project work has been the driving force in many of their research 
activities. This points towards the important relationship between research and development. 

The relationship between research and development 

Instructional Technology is an active field, and it is important to remember that in the final analysis it is our 
job to make a positive impact in the way that people learn. Saettler (1990) reminds us that technology in our field is 
defined quite broadly. We work with hard technologies, sometimes referred to as “things that plug in ” but we also 
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work with soft technologies, or “advances in thought.” Always, we are ibokfhg forward. Though much of what we 
do is practical in nature, even studies in seemingly esoteric areas such as post modernism are used to inform our 
work. In our field, research certainly influences practice. However, this is a reciprocal arrangement because often 
times in instructional technology practice influences research. 



Blurring lines 

Most research leaders find that the boundaries between work interests and research interests blur 
significantly. Many researchers with advanced degrees have worked for years before returning to study for an 
advanced degree and pursuing a career that involves research. Because of this, many of their initial research 
questions cam from their work experiences. As they continued to do research, they also continued to do work in the 
field. Some of that work may have been in the design and/or the development of educational programs. For many 
of our interviewees, they were involved in significant evaluation activities. While the type of work may vary, what 
remains constant is the connection between work experiences and research projects. Many researchers may serve 
dual roles. They may design the environment conduct research on both the process of building the environment and 
the ultimate value of the finished environment. This indicates a very strong connection between research and 
development, and suggests that it is not only common for researchers in instructional technology to also be designers 
and developers of instructional products, but that it is also highly desirable. 

One researcher recommended that working on projects was so important that young academics should do it 
for free if need be. The practical applications of this work can greatly improve your skills as a researcher and keep 
you in touch with what the field is doing. So while most leaders in research tend to credit the real life experience 
they have, they recognize it as not only important in giving them experience, but in keeping them current as well. 

Connecting research and practice 

Combing research and development may be desirable, but what can be difficult is making the results of 
your research valuable to others. We face a problem with research in our field according to our researchers. Chiefly 
we don’t connect practitioners to research as seamlessly as we might desire. It is often assumed that the role of a 
researcher is to tell the practitioner what to do. However, people who are working in the field everyday have many 
ideas that can make interesting research questions, or may have answers to many significant questions in our field. 
In instructional technology, many active researchers are also practicing professionals, but the connection between 
practice and research should be made stronger. It has been suggested by some leaders that our research efforts 
should be focused on making information available to practitioners in schools, industrial, or military applications. 
Our research, and the results of our research, need to be more accessible to practitioners. Our research, and our 
field, needs to make a greater impact on the environments in which we work; our research results need to be more 
practical. 



The most important piece of work 

Finding interesting work and letting it guide your research can make your research not only more valuable 
to the field, but more interesting to yourself as well. Many of our interviewees are remembered for a “seminal 
work” or a particular theory or line of inquiry that they have been working on for a period of time. However nearly 
all of them expressed that the most interesting thing that they had done was what they were working on now. This 
indicates a passion for research, and a commitment to new ideas. It would be hard to be involved in a project if you 
were not interested in it. Therefore one recommendation made is that you work to find projects that can hold your 
interest. 

Doing research on your work is a piece of advice that is often heeded by many people. Working in the 
field is both professionally and personally rewarding. Because many of the researchers point out that we need to be 
studying naturally occurring environments to see how people learn, working in environments becomes even more 
critical. Finding a naturally occurring environment in which you can work is not easy. Establishing yourself by 
working on a project, or in an educational environment can make research populations available to you. 

Choosing a mentor 

There is much to know about research. There is the problem of identifying questions, seeking funding, 
managing the process, and ultimately disseminating your work and writing for publication. One of the most 
important things a novice researcher can do is to seek out a mentor to help them learn and master this process. The 



definition of mentor here is not limited to the relationship between a student'and a professor in graduate school. A 
novice, researcher is well advised to seek out others in the field who can help them work in the area of research. 
Many leaders advise not only the novice to search out mentors, but also the mentors to treat their apprentices in a 
collegial manner. 

The role of the mentor 

Mentors can do a lot for an apprentice. Beyond simply teaching them how to do things, they can introduce 
them to people in the field, or involve them in research and writing projects. Apprentices gain valuable experience 
in writing for publication from their mentors. But ultimately what the mentor can offer to an apprentice is 
confidence, and confidence is one of the most important skills one can develop as a researcher. To submit your 
work to the rigors of the referee process, one needs to have a bit of an ego, and a thick skin. Mentors can help 
novice researchers develop these in the right amount and appropriate manner. One researcher relates a story of how 
they were “cooked” good by a discussant at a conference. The presentation went abysmally, but ultimately the 
paper was reworked and published. The research, ideas, and the paper had value, but that value wasn’t recognized 
during the first presentation of the idea. It was not until sometime later that the paper made it into publication. The 
point is that simply because one person does not like or agree with a research idea it does not mean that the idea 
should not be pursued. Mentors can help apprentices see this and gain the skills and confidence they need to 
continue working when an idea is rejected. 

The role of the apprentice 

An apprentice can offer much to the mentor as well. Often times, it is an apprentice that gets their mentor 
involved in a particular project or line of research. One suggested reason for this is that mentors work with more 
and more apprentices. While a mentor may have spent their time reading and writing in the past, more and more of 
their time is spent working with more and more people. It becomes a challenge to stay abreast of everything that is 
going on in the field. Often times it is the apprentice, who can focus their efforts and energies more narrowly, who 
can point the mentor in a new direction. The point here is one of collegiality. It is generally recognized that people 
working together can bring diverse views, skills, and ideas to a project. The same is true of the relationship between 
the mentor and the apprentice. Care should be taken in choosing a mentor. You should choose a mentor who not 
only publishes, but who publishes with their apprentices. Novice researchers, or people in their first academic 
position, should choose a mentor where they work who can help them find resources, locate outlets for publishing, 
and help them learn to write grants. 

Most people recognize that the mentor has much to offer the apprentice. It is suggested here that the 
apprentice also has much to offer the mentor. This reciprocal relationship promotes not only a greater understanding 
of the field to the novice researcher, but also helps keep the mentor current in working with others in the field. 

One caveat- one of our interviewees points out that there simply may not be enough mentors to go around. 
Mentoring takes a lot of time and entails a special relationship between the faculty and students. Given the fairly 
large student faculty ratio in many of our programs it may not be possible for every student to have a mentor. In 
fact, it may be the case that if your are fortunate enough to have a mentor, then you are fortunate indeed. 

A question of methods 

The issue of whether to do qualitative or quantitative research is one that most researchers are faced with. 
Historically, much of the research that was done in education in general and instructional technology specifically, 
was quantitative in its planning and execution. For many years, faculty supervising research were trained solely in 
statistical research methods. Today, more and more researchers are looking to alternative research methodologies 
such as qualitative research. 

Research traditions 

Much of our research methodology has been derived from the hard sciences. There are many reasons for 
this. Historically the field of education has constructed controlled environments in which to test single or multiple 
variables. Most often these were contrived environments, and if you look at research in the field of instructional 
technology you will see that most of the research in the past several years still follows this paradigm (Reeves, 1995). 

Many researchers interviewed here state that their initial research involved isolating single variables and 
working with them in a controlled environment. One reason for this may be that many of our research leaders 
originally were psychologists, or educational psychologists trained in the dominant research paradigm of the times. 



The dominant research paradigm for many years has been quasi-experimental studies. People researched in a 
manner in which they were taught to research. Some leaders suggest that much of their initial research agenda, 
while interesting to them, was also driven by the tenure and promotion process. A combination of history and 
environment has driven the field for years in the types of methods and research tools it uses. Currently, most leaders 
are recognizing that there are more tools available to researchers today and are seeking to use these tools in research. 

Mixed methods 

A recurring message from nearly all our research leaders is to avoid the qualitative versus quantitative 
debate entirely. This debate has been characterized as “mindless” by some people. Our research history, and much 
of our current research recognizes the need to isolate variables. However much of our current research also realizes 
that variables in the real world are not really controllable, and they rarely behave like the variables that are able to be 
controlled in the world of physical science. Most of the people interviewed here work with the philosophy that in 
order to answer certain questions you will need certain sets of tools, and therefore suggest that researchers study and 
develop skills in both qualitative and quantitative research methods. When an environment presents itself you will 
then be ready to study it regardless of what questions may need to be asked, or what tools are needed to answer the 
questions. 

Additionally, some of the leaders interviewed here suggest that beyond qualitative and quantitative research 
methods, there exists the need to consider issues in yet more divergent research paradigms. Among these are the 
methods associated with post modernism, semiotics, and historical research. Additionally there are at least two 
researchers who make the argument that our research not only needs to be more responsive to practitioners, but that 
it needs to be more responsive to society as well. There is small, but growing call in the field for socially 
responsible research in instructional technology. 

Disseminating research 

Research is done to inform practice, and the results of research are meant to be shared with the larger 
community. Disseminating research begins with having questions that need answering, and culminates in the 
publication of that research. 

The research agenda 

Traditionally research is disseminated in two formal ways: the conference presentation and the journal 
article. Informally the results of research can be spread through informal conversation among colleagues. People 
need to share the results of their research so that they can not only make sense out of what they are doing, but also so 
that may gain insight into the next phase of the research, or the next study in a long term research agenda. Many 
people speak of research as a pipeline. You conduct work on an idea that gets proposed to a conference. The initial 
paper is done and presented at a conference. Based on reactions and feedback at the conference the paper is 
reworked for submission to a refereed journal. While you are working on one project, you should be considering 
other issues that you can work on later. Work that can be proposed to conferences the next year will keep you busy 
after one piece goes in to print. The goal is to keep the pipeline full at all times, meaning that there should always 
be a project to work on. For those in a tenure earning position, it should be noted that the tenure process is too short 
to begin work in distinctly different areas, but that rather you should focus your work within a common framework. 
This common framework, or area of research is often referred to as a research agenda. 

For most researchers, the research agenda will start with the dissertation or thesis and move forward from 
there. While many researchers can and do make dramatic shifts in their research interests, most tend to work in a 
broad area such as evaluation, computer-based learning environments, electronic performance support systems, 
cross cultural research, or the processes associated with instructional design to name but a few. The key point is that 
by staking out an area in which to work a researcher does a couple of significant things. They focus on key 
problems and become recognized as a person who can inform this small area of the field. Taken in concert with 
other people working in either similar areas, or in completely different areas, a fabric is woven that constitutes the 
body of research in the field. By establishing a research agenda, or a line of inquiry, a researcher gets to the 
business of making a contribution by advancing the knowledge of the field. As you become recognized as an expert 
in that area, you can begin to help others conduct research in that area, and you can begin to make connections 
between your research area and other research areas. For example a person with significant research experience in 
distance education could team up with a person with significant experience in hypertext to conduct research in web- 



based instruction. The two areas inform each other, and can be used to jndve forward into a new area. The two 
areas do not represent mere appurtenances to each other, but rather represent unique applications of divergent areas. 

Writing research 

People in the “real world” are interested in our work, but they find it hard to put their hands on the 
information they need easily. This is suggested by more than one researcher. Our field is filled with three letter 
acronyms (TLA’s) that are often seen as cliquish in nature and make it sometimes more difficult for people to use 
the results of our research than we would intend. While it raises the question of “who do you write for, it may also 
suggest that rather than saying things to sound intellectual, we should simply say what we mean. We should avoid 
confusing language, or, as the bumper sticker says: "Eschew obfuscation." 

It is a recurring theme in nearly all of the interviews that one of the biggest problems we face as a field is in 
having our work make a difference in the life of the individual, and that one of the ways in which we can achieve 
this is through the dissemination of our research. Clarity of writing is stressed, as is the age-old soul of wit, brevity. 

Conferences and journals 

Conference presentations and journal articles remain the two most common ways to disseminate research. 
While not mentioned in the interviews directly, it is well known that the lag time between writing and publication 
can be fatal to a new idea: by the time the new idea gets published the idea is already generally recognized as being 
obsolete. Most of us will have to begin planning what our conference presentations will be so long before the 
conference even begins that some ideas may be found to be inappropriate by the time the conference arrives. 

One reason that it can take so long for research results to be published is the referee process. It does take 
time for busy people to read their colleagues’ work carefully and make decisions on its appropriateness for 
publication. Web-based publishing has been mentioned by more than one researcher as an alternative to the 
traditional paper based journals. While changing the medium itself is not likely to yield dramatic results, it is 
possible that it could speed up both the publication speed and help increase access to the published research. 

Conclusions 

Hannafin and Hannafin (1995) warn that in the future IT might "continue its emphasis on training 
practitioners, but will be unable to forge its own destiny and advance its own research.” (P. 320) It is important that 
we pay greater attention to research and to training researchers if we are to avoid this somewhat dire consequence. 
Becoming a researcher is not the same thing as learning to do research. We can learn research methodology in our 
classes on quantitative and qualitative analysis. We can learn to do literature reviews through research seminars. 
But knowing these things does not guarantee that one will be able to put them together in an intelligent, scholarly, 
and creative fashion. Only by working with those among us who are already doing quality research, can we began to 
discover how to synthesize these elements in such a fashion as to do quality research ourselves. There may not be 
enough mentors to go around. But we hope that through the interviews with the research leaders on InTRO we can 
assist those mentors among us in reaching and in teaching enough of us to make a difference. 
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